In order to more effectively combine the detail information of high resolution PANchromatic (PAN) image and the spectral information of low resolution MultiSpectral (MS) image, an improved panchromatic sharpening algorithm based on sparse representation was proposed. Firstly, the intensity channel of an MS image was down-sampled and then up-sampled to get its low-frequency components. Secondly, the MS image intensity channel minus low-frequency components to obtain its high-frequency components. Random sampling was performed in the acquired high and low frequency components to construct a dictionary. Thirdly, the PAN image was decomposed to get the high-frequency components by using the constructed overcomplete dictionary. Finally, the high-frequency components of the PAN image were injected into the MS image to obtain the desired high-resolution MS image. After a number of experiments, it was found that the proposed algorithm subjectively retains the spectral information and injects a large amount of spatial details. Compared with component substitution method, multiresolution analysis method and sparse representation method, the reconstructed high resolution MS image by the proposed algorithm is more clear, and the correlation coefficient and other objective evaluation indicators of the proposed algorithm are also better.